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Objectives: Although sonography-guided core needle biopsy is a highly targeted method, there is a possibility of an invasive 
component after surgical excision of ductal carcinoma in situ (DCIS) of the breast. This study was performed to develop 
and validate nomograms to predict the postoperative invasive component in DCIS at core needle biopsy Methods: Two 
nomograms were developed using the data of previous meta-analysis and multivariate analysis. Nomograms were validated 
externally using the data of the authors' affiliation. The accuracy was validated by the expected-to-observed ratio and the 
Hosmer-Lemeshow goodness-of-fit test. Discrimination was validated by the area under the curve (AUC) of receiver operat- 
ing characteristic (ROC) curve analysis. Results: The nomogram using the meta-analysis study data was developed at http:// 
dcis-m.surgery.kr.pe/, and the nomogram using the multivariate analysis study data was developed at http://dcis-k.surgery. 
kr.pe/. The Hosmer-Lemeshow goodness-of-fit test showed that the nomogram using multivariate analysis data (p = 0.131) 
was better calibrated than that using meta-analysis data (p < 0.001). ROC curve analysis showed statistically significant power 
of discrimination in both nomograms (AUC = 0.776, 0.751). Conclusions: Both nomograms showed statistically significant 
discriminatory power, but the nomogram using the data of multivariate analysis was simpler and more reliable. These would 
be useful for the prediction of invasive cancer and the need for sentinel node biopsy in DCIS at core needle biopsy. 
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I. Introduction 

Sonography-guided core needle biopsy is a standard tool for 
the diagnosis of breast cancer including ductal carcinoma in 
situ (DCIS). However underestimation is possible because 
only a small part of the tumor is sampled. After complete 
excision, the invasive component can be found in the final 
pathologic diagnosis of DCIS. In the case of a postoperative 
diagnosis of invasive cancer, additional axillary biopsy is 
needed because axillary metastasis is possible. Therefore, in 
cases in which the risk of invasive cancer is high, simultane- 
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ous axillary biopsy with surgery of the primary tumor can 
avoid the need for additional surgery. 

There have been many studies on the prediction of postop- 
erative invasive cancer in the case of preoperative DCIS. The 
meta-analysis of 52 studies of DCIS at core needle biopsy 
showed that symptomatic presentation, palpability, size, 
mammographic mass, Breast Imaging-Reporting and Data 
System (BI-RADS) category, biopsy method, and histologic 
grade were significant factors in the underestimation of 
DCIS [1], 

In this study, we developed two Web-based nomograms 
to predict the postoperative invasive component in DCIS at 
core needle biopsy using the existing references of a meta- 
analysis study [1] and a multivariate analysis study [2] and 
compared the two nomograms in terms of validation of reli- 
ability and discrimination using the data of the authors' af- 
filiation. 

II. Case Description 

1. Estimation of Model using Existing References 

A nomogram was developed by using a multivariate linear 
logistic regression model. The regression coefficient (p) was 
calculated using the odds ratio (OR). The model was esti- 
mated as follows: 

P = log(OR), 
logit P = + p 2 X 2 + - + PiXi 

The two models were selected using data of the meta-analy- 
sis study [1] and the multivariate analysis study [2]. Selected 
meta-analysis study was the only study of that kind reported 
prior to the period of research, and we selected the largest 
Korean study among the many multivariate analysis studies. 
The risk factor selected were those that were statistically sig- 
nificant. The meta-analysis data [1] included a total of 7,350 
cases of DCIS. Among them, 1,736 cases were diagnosed as 
invasive cancer finally. The model of meta-analysis study can 
be written as, 

logit P of meta-analysis study = log(2.57) x 
symptomatic presentation + log(3.87) x 
palpability + log(2.28) x size more than 20 
mm + log(1.83) x mammographic mass + 
log(0.33) x BI-RADS category 3 + log(1.46) 
x BI-RADS category 5 + log(0.54) x 
11 -gauze device + log(0.56) x low or inter- 
mediate grade. 



Nomogram for Postoperative Invasion in DCIS 

The multivariate analysis study [2] showed 216 cases of 
invasive cancer among 506 cases of DCIS. The model of the 
multivariate analysis study is given as 

logit P of multivariate analysis study = 
log(1.89) x palpability + log(1.02) x size + 
log(1.69) x 14-gauze device + log(1.71) x 
high grade. 

The probability of underestimation was calculated as in [3] 
as 

Probability of underestimation = T — t— 

1 + e -logitP 

2. Calibration of Regression Models using the Risk Intercept 

The average rate of invasive cancer was 25.9% in the meta- 
analysis study and 42.7% in the multivariate analysis study. 
The rate of invasive cancer varied according to the target 
population; therefore, calibration of the model was manda- 
tory. The risk intercept was defined as the rate of no risk fac- 
tors divided by the rate of average risk factors. The rate of no 
risk factors was calculated as the summation of probability 
with no risk factors. The rate of the average risk was calcu- 
lated as the summation of the probability x frequency of the 
risk factor combinations. 



Risk intercept (%) = 1+ f 

£ 1+e -l°gitP 

The intercept a was calculated as 

Risk intercept (%) x Observed rate = - — - 



Risk intercept x Observed rate 

3. Development of the Web-Based Nomograms 

Each Web-based nomogram consisted of an HTML file 
which input the risk factors and a CGI file which calculated 
and output the results. The nomogram using the meta- 
analysis study data was developed at http://dcis-m. surgery, 
kr.pe/. The nomogram using the multivariate analysis study 
data was developed at http://dcis-k.surgery.kr.pe/. The de- 
fault values of the average rate of invasive cancer were 25.9% 
in the meta-analysis data [1] and 42.7% for the multivariate 
analysis data [2] based on the references. Values of the aver- 
age rate of invasive cancer were entered by input form and 
the values were modifiable. The output part included each 
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OR according to the risk factors and the expected rate of in- 
vasive cancer. 

4. Validation of the Nomograms 

From January 2006 to June 2013, patients diagnosed as hav- 
ing DCIS by sonography-guided core needle biopsy were 
selected for the validation of the nomograms. The reliability 
of each nomogram was validated by comparing the expected 
number (E) with the observed number (O) of invasive can- 
cers. The expected-to-observed (E/O) ratio was calculated 
according to each risk factor. Here, the 95% confidence in- 
tervals (CI) of the E/O ratio were calculated as 

95% CI for E/O ratio = -e"' 96 *^ 
o 

The correlation between the expected number and the 
observed number of invasive cancers was confirmed by the 
Hosmer-Lemeshow goodness-of-fit test. The discrimination 
of the nomograms was validated by the area under the curve 



Table 1. Factors of postoperative invasive cancer in the valida- 
tion data 





Invasive cancer 


Total 




No (n = 40) Yes (n = 24) 


(n = 64) 


Presentation 








Screening detected 


27 (69) 


12 (31) 


39 


Symptomatic 


13 (52) 


12 (48) 


25 


Palpability 








No 


29 (71) 


12 (29) 


41 


Yes 


11 (48) 


12 (52) 


23 


Size (mm) 








<20 


29 (85) 


5(15) 


34 


>20 


11 (37) 


19 (63) 


30 


Mammographic mass 








No 


29 (81) 


7(19) 


36 


Yes 


11 (39) 


17(61) 


28 


BI-RADS category 








4 


35 (60) 


23 (40) 


58 


5 


5 (83) 


1(17) 


6 


Biopsy method 








14-gauze 


24 (55) 


20 (45) 


44 


11 -gauze 


16 (80) 


4(20) 


20 


Grade 








High 


8 (35) 


15 (65) 


23 


Low or intermediate 


32 (78) 


9 (22) 


41 



Values are presented as number (%). 

BI-RADS: Breast Imaging-Reporting and Data System. 
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(AUC) of receiver operating characteristic (ROC) curve 
analysis. We compared the two nomograms using MedCalc 
ver. 12.7 (MedCalc Software, Ostend, Belgium). 

A total of 64 cases of preoperative DCIS were included in the 
validation data set. Among them, 24 cases (37.5%) were diag- 
nosed as invasive cancer postoperatively, so we used 37.5% as 
the average rate of invasive cancer. The clinical, radiological, 
and pathological factors are summarized in Table 1. 

In Table 2, the E/O ratio and 95% confidence intervals are 
described from the nomogram using the meta-analysis study 
data. Overall, the nomogram showed a tendency of under- 
estimation, but the tendency was not statistically significant. 
In the subgroup of screen detected, non-palpable mass, and 
size of more than 20 mm, the nomogram showed statistically 
significant underestimation. 

In Table 3, the same confidence interval was described from 
the nomogram using the multivariate analysis study data. 
Overall, it showed a tendency of overestimation, but the ten- 



Table 2. Expected-to-observed ratio and 95% CI according to the 
factors of invasive cancer from the meta-analysis data 





Observed 


Expected 


E/O ratio 




(0) 


(E) 


(95o/o CI) 


Overall 


24 


17.05 


0.71 (0.48-1.06) 


Presentation 








Screening detected 


12 


3.75 


0.31 (0.18-0.55) 


Symptomatic 


12 


13.30 


1.11 (0.63-1.95) 


Palpability 








No 


12 


3.97 


0.33 (0.19-0.58) 


Yes 


12 


13.08 


1.09 (0.62-1.92) 


Size (mm) 








<20 


5 


5.60 


1.12 ( 0.47-2.69) 


>20 


19 


11.45 


0.60 (0.38-0.94) 


Mammographic mass 








No 


7 


4.71 


0.67 (0.32-1.41) 


Yes 


17 


12.33 


0.73 (0.45-1.17) 


BI-RADS category 








3 


0 


0.00 




4 


23 


15.47 


0.67 (0.45-1.01) 


5 


1 


1.58 


1.58 (0.22-11.22) 


Biopsy method 








14-gauze 


20 


15.54 


0.78 (0.50-1.20) 


11 -gauze 


4 


1.51 


0.38 (0.14-1.01) 


Grade 








High 


15 


9.84 


0.66 (0.40-1.09) 


Low or intermediate 


9 


7.21 


0.80 (0.42-1.54) 


CI: confidence interval, BI-RADS: Breast Imaging; 


[-Reporting and Data 



System. 
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Table 3. Expected-to-observed ratio and 95% CI according to the 
factors of invasive cancer from the multivariate analysis data 





Observed Expected 


F/O rutin 

L/ IdLIU 






fFl 


fQCiO/n nil 


\J VCI dll 


24 


34.42 




r dipdDlllLy 








INQ 


1 0 


lo./o 


1 ^£ QQ 0 TO 

loo ^u.oy— z./Dj 


ICS 


1 1 


ID. DO 


i ii ha i iri\ 


Size (mm) 








/in 

szu 


r 
D 


1 A £3 


1 Q"2 / 1 T> H flQ^ 

z.y3 l.ZZ— /.U3J 




19 


19.80 




Biopsy method 








11 -gauze 


4 


7.88 


1.97 (0.74-5.25) 


14-gauze 


20 


26.54 


1.33 (0.86-2.06) 


Grade 








Low or intermediate 


9 


19.08 


2.12 (1.10-4.08) 


High 


15 


15.34 


1.02 (0.62-1.70) 



CI: confidence interval. 
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Nomogram from meta-analysis data 

Total patients per decile 

- Observed number of invasive cancer patients 

- Expected number of invasive cancer patients 




4 5 6 
Deciles of patients 

Figure 1. Hosmer-Lemeshow goodness-of-fit test of nomogram 
from the meta-analysis data (p < 0.001). Expected 
number of invasive cancer patients, observed number 
of invasive cancer patients, and the corresponding 
number of total patients per decile are shown. 

dency was not statistically significant. In the subgroup of size 
less than or equal to 20 mm and low or intermediate grade, 
the nomogram showed statistically significant overestima- 
tion. 

Figures 1 and 2 show the results of the Hosmer-Lemeshow 
goodness-of-fit test. The nomograms using multivariate 
analysis data (p = 0.131) showed better calibration than that 
using meta-analysis data (p < 0.001), so the nomogram using 
multivariate data was more reliable than the nomogram us- 
ing meta-analysis data. 
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Nomogram from multivariate analysis data 

Total patients per decile 

Observed number of invasive cancer patients 

Expected number of invasive cancer patients 




2 
0 

123456789 
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Figure 2. Hosmer-Lemeshow goodness-of-fit test of nomogram 
from the multivariate analysis data (p = 0.131). Expect- 
ed number of invasive cancer patients, observed number 
of invasive cancer patients and the corresponding num- 
ber of total patients per decile are shown. 
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Figure 3. Receiver operating characteristic curves of nomograms 
from the meta-analysis data and multivariate analysis 
data. The area under curve is 0.766 (p < 0.001) for the 
meta-analysis data and 0.751 (p = 0.001) for the multi- 
variate analysis data. The curves show no differences (p = 
0.614). 

In the validation of discrimination power, the AUC of the 
nomogram using the meta-analysis study data was 0.766 
(95% CI, 0.650-0.882; p < 0.001), while the AUC of the no- 
mogram using multivariate analysis study data was 0.751 
(95% CI, 0.628-0.873; p = 0.001). The ROC curves of both 
nomograms showed no difference in discrimination (p - 
0.614) (Figure 3). 

III. Discussion 

In this study, we developed two nomograms: one using the 
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ORs of a meta-analysis study and the other using the ORs of 
a multivariate analysis study. The results showed that the two 
nomograms achieve similar discriminatory power. How- 
ever, the nomogram using the multivariate analysis data is 
simpler and more reliable than that using the meta-analysis 
data. This may be due to the correlations between factors. 
Meta-analysis is a univariate analysis, and prediction can be 
incorrect if factors correlate with each other. We confirmed 
that the nomogram using the meta-analysis data was more 
complicated and inaccurate. 

A nomogram can be validated by both internal and external 
validation. Internal validation uses the data of the same pop- 
ulation for the development of the nomogram, and external 
validation uses the data of a different population. In most 
studies, internal validation is performed first. However, most 
nomograms are used in populations that are different from 
the one that was used for the development of the nomogram; 
therefore, inaccurate results are possible. For example, no- 
mograms developed in a western country showed decreased 
accuracy when applied to oriental people [4]. We used a 
different data set for the external validation, which showed 
significant reliability and discrimination in the nomogram 
using the multivariate study data; therefore, it would be pos- 
sible to use it in clinical situations. 

For more accurate prediction, we developed a nomogram 
with an intercept calibration and a changeable rate of post- 
operative invasive cancer. Showing great variety, the rate of 
postoperative invasive cancer has been reported in the range 
between 0% and 59% [1], and the rate varies according to 
each affiliation. Our nomograms can change the rate, and 
this should make accurate prediction more likely. 

In conclusion, we developed two nomograms based on 
studies of meta-analysis and multivariate analysis. Both 
nomograms showed statistically significant discriminatory 
power, but the nomogram using the multivariate analysis 
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data was simpler and more reliable. The nomogram using 
the multivariate analysis data would be useful for the predic- 
tion of invasive cancer and the need for sentinel node biopsy 
in DCIS at core needle biopsy. 

Conflict of Interest 

No potential conflict of interest relevant to this article was 
reported. 

Acknowledgments 

This research was conducted by the research fund of 
Dankook University in 2014. 

References 

1. Brennan ME, Turner RM, Ciatto S, Marinovich ML, 
French JR, Macaskill P, et al. Ductal carcinoma in situ 
at core-needle biopsy: meta-analysis of underestima- 
tion and predictors of invasive breast cancer. Radiology 
2011;260(l):119-28. 

2. Kim J, Han W, Lee JW, You JM, Shin HC, Ahn SK, 
et al. Factors associated with upstaging from ductal 
carcinoma in situ following core needle biopsy to in- 
vasive cancer in subsequent surgical excision. Breast 
2012;21(5):641-5. 

3. Iasonos A, Schrag D, Raj GV, Panageas KS. How to build 
and interpret a nomogram for cancer prognosis. J Clin 
Oncol 2008;26(8):1364-70. 

4. Kim SH, Chae YS, Son WJ, Shin DJ, Kim YM, Chang 
MC. Estimation of individualized probabilities of devel- 
oping breast cancer for Korean women. J Korean Surg 
Soc 2008;74(6):405-ll. 



156 www.e-hir.org 



http://dx.doi.Org/l0.4258/hir.2014.20.2.152 



